fix: UDF default return_field_from_args correctly computes nullability from input arguments #19213
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Yuvraj-cyborg
force-pushed
the
ascii_nullability
branch
2 times, most recently
from
47933b1 to
acdbf72
Compare
Yuvraj-cyborg
force-pushed
the
ascii_nullability
branch
from
acdbf72 to
2b6ab6e
Compare
Yuvraj-cyborg
force-pushed
the
ascii_nullability
branch
from
2b6ab6e to
38cfd10
Compare
Contributor
Author
|
@rluvaton hey, could you review it please ! |
Member
|
I think changing here is too of a breaking change. |
Contributor
I think this makes sense to have, as I see the multiple PRs being raised to fix individual (Spark) functions and can see not having this as default makes it not very ergonomic for downstream functions; that said, we'd probably need some more investigation to make sure affected functions (i.e. all of them...) will play well with this change. Perhaps we can raise a separate issue for this entirely (e.g. "Change default nullability of UDFs") so can have more visibility on the discussion? |
Contributor
|
I've raised #19511 for discussion We won't proceed with the changes in this PR |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #19169 - spark
asciineeds custom nullability.Rationale for this change :
The current implementation of date_add UDF does not correctly handle nullability for its return type. This change ensures that the return type of the date_add function is nullable, following the expected null propagation behavior for SQL functions.
Changes in PR:
Test Coverage:
Yes, tests are included to verify the change.
User-facing Changes:
No user-facing changes.